Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add alpine docker image definition #88

Merged
merged 11 commits into from
Nov 13, 2019
Merged

Add alpine docker image definition #88

merged 11 commits into from
Nov 13, 2019

Conversation

vvmruder
Copy link
Contributor

Just a suggestion

@vvmruder vvmruder changed the title add alpine docker image definition [WIP] add alpine docker image definition Oct 31, 2019
@tcommandeur tcommandeur changed the base branch from master to development November 4, 2019 08:49
@vvmruder
Copy link
Contributor Author

vvmruder commented Nov 5, 2019

@balazsdukai From my point of view this should work now. I tried to squeeze it as much as possible. But couldn't get image smaller than 1.4gb 😢
You might try it now. It lasts a while to build locally. But it includes spatialite support (I had some chat with someone I know... and now it works) even with proj6 and gdal 3. Iam specially interested how it works for your flow using PostGIS. So: Let me know.

@vvmruder
Copy link
Contributor Author

vvmruder commented Nov 5, 2019

Since there is already a travis configuration I was not editing it. But maybe we should also update this like in the other projects.

@vvmruder
Copy link
Contributor Author

vvmruder commented Nov 5, 2019

And: I will update the readme soon.

@balazsdukai
Copy link
Member

Great, thanks! I'm gonna test it and merge. I'll also update the travis build script I think, it'll be useful.
Yeah I guess 1,4 gb it is. The size makes sense I think since it includes the whole OSGeo stack of libraries :-) .

@vvmruder
Copy link
Contributor Author

vvmruder commented Nov 6, 2019

Yes. I tried to offer as much input as possible for vector features.

You are aware of the "master" version of spatialite? Iam convinced it will work for the moment. But still, it is master...

@balazsdukai
Copy link
Member

Hm no, what's the "master" version of spatialite? :-)

@vvmruder
Copy link
Contributor Author

vvmruder commented Nov 6, 2019 •
edited
Loading

Its basically the current state of development. So it potentially contains pre alpha code. But as far as I understood the guys maintaining spatialite are really careful about working master branch. So in the moment this should be ok. Was just a side note.

@balazsdukai
Copy link
Member

@vvmruder I'm building the image but I get a connection time out for download.osgeo. Did you ever got that?

(17/18) Installing unzip (6.0-r4)

(18/18) Installing .proj4-deps (20191107.131458)

Executing busybox-1.30.1-r2.trigger

OK: 174 MiB in 40 packages

Connecting to download.osgeo.org (140.211.15.30:80)

wget: can't connect to remote host (140.211.15.30): Operation timed out

Error: ResponseItem.ErrorDetail[code=1,message=The command '/bin/sh -c apk --update add sqlite libstdc++ sqlite-libs libgcc &&     apk --update add --virtual .proj4-deps         make         gcc         g++         file         sqlite-dev         unzip &&     cd /tmp &&     wget http://download.osgeo.org/proj/proj-${PROJ_VERSION}.tar.gz &&     tar xfvz proj-${PROJ_VERSION}.tar.gz &&     rm -f proj-${PROJ_VERSION}.tar.gz &&     wget http://download.osgeo.org/proj/proj-datumgrid-1.8.zip &&     unzip proj-datumgrid-1.8.zip -d proj-${PROJ_VERSION}/nad/ &&     rm -f proj-datumgrid-1.8.zip &&     wget http://download.osgeo.org/proj/proj-datumgrid-europe-1.1.zip &&     unzip proj-datumgrid-europe-1.1.zip -d proj-${PROJ_VERSION}/nad/ &&     rm -f proj-datumgrid-europe-1.1.zip &&     wget http://download.osgeo.org/proj/proj-datumgrid-north-america-1.1.zip &&     unzip proj-datumgrid-north-america-1.1.zip -d proj-${PROJ_VERSION}/nad/ &&     rm -f proj-datumgrid-north-america-1.1.zip &&     wget http://download.osgeo.org/proj/proj-datumgrid-oceania-1.0.zip &&     unzip proj-datumgrid-oceania-1.0.zip -d proj-${PROJ_VERSION}/nad/ &&     rm -f proj-datumgrid-oceania-1.0.zip &&     cd proj-${PROJ_VERSION} &&     ./configure &&     make &&     make install &&     echo "Entering root folder" &&     cd / &&    echo "Cleaning dependencies tmp and manuals" &&     apk del .proj4-deps &&     rm -rf /tmp/* &&     rm -rf /user/local/man &&     for i in /usr/local/lib/libproj*; do strip -s $i 2>/dev/null || /bin/true; done &&     for i in /usr/local/lib/geod*; do strip -s $i 2>/dev/null || /bin/true; done &&     for i in /usr/local/bin/proj*; do strip -s $i 2>/dev/null || /bin/true; done &&     proj' returned a non-zero code: 1]

@vvmruder
Copy link
Contributor Author

vvmruder commented Nov 7, 2019

Tried to reach it in browser. But it seems really lagging. So probably they have some problem with their servers?

@balazsdukai
Copy link
Member

Yes, seems so.

@balazsdukai
Copy link
Member

I also get a warning about Boost when building 3dfier. Do you get this @vvmruder ?

CMake Warning at /usr/share/cmake/Modules/FindBoost.cmake:905 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FindBoost.cmake:1027 (_Boost_COMPONENT_DEPENDENCIES)
  /usr/share/cmake/Modules/FindBoost.cmake:1727 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:24 (find_package)


CMake Warning at /usr/share/cmake/Modules/FindBoost.cmake:905 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FindBoost.cmake:1027 (_Boost_COMPONENT_DEPENDENCIES)
  /usr/share/cmake/Modules/FindBoost.cmake:1727 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:24 (find_package)


CMake Warning at /usr/share/cmake/Modules/FindBoost.cmake:905 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FindBoost.cmake:1027 (_Boost_COMPONENT_DEPENDENCIES)
  /usr/share/cmake/Modules/FindBoost.cmake:1727 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:24 (find_package)


CMake Warning at /usr/share/cmake/Modules/FindBoost.cmake:905 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FindBoost.cmake:1027 (_Boost_COMPONENT_DEPENDENCIES)
  /usr/share/cmake/Modules/FindBoost.cmake:1727 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:24 (find_package)


CMake Warning at /usr/share/cmake/Modules/FindBoost.cmake:905 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FindBoost.cmake:1027 (_Boost_COMPONENT_DEPENDENCIES)
  /usr/share/cmake/Modules/FindBoost.cmake:1727 (_Boost_MISSING_DEPENDENCIES)
  CMakeLists.txt:24 (find_package)


-- Boost version: 1.71.0

@vvmruder
Copy link
Contributor Author

Yes. This warnings I had too. I was intentionally ignoring them because whole processes run. But I had to use this new boost version because of proj6. This is a topic we can discuss. I thought it might be a good idea to go with some up to date versions. But maybe we should stick to proj4. This would make use of 'older' versions of involved libraries possible.

@balazsdukai
Copy link
Member

The process runs for me too, as least seemingly. However, 3dfier doesn't generate any output. It says it read the point clouds, but I'm pretty sure it didn't, because 3dfier finishes waay to quickly. Did you get that issue?

docker run --rm -v "$(pwd)":/data tudelft3d/3dfier:latest 3dfier testarea_config_unix.yml --CityJSON testarea.json

@vvmruder
Copy link
Contributor Author

I dar to ask. But: Are the files defined as full access on the host machine? Since we use random user it need to be configured like this. And I recognized that 3dfier fails silently if there are permission problems with the configured files.

Which OS you are using?

@vvmruder
Copy link
Contributor Author

image

I always give this permission pattern on all folders and files recursively to make 3dfier working correctly.

@balazsdukai
Copy link
Member

balazsdukai commented Nov 11, 2019 •
edited
Loading

But read should be sufficient, no?

-rw-rw-r-- 1 balazs balazs  3532883 jul 27  2018 ahn3_cropped_1.laz
-rw-rw-r-- 1 balazs balazs  5836325 jul 20  2018 ahn3_cropped_2.las
-rw-rw-r-- 1 balazs balazs  1170107 jul 27  2018 ahn3_cropped_2.laz

But in any case, I just tried with o+rwx for all input files and directory. Still the same result (no result).

@vvmruder
Copy link
Contributor Author

That's strange. Are you using it with PostGIS? What is the output of 3dfier?

@balazsdukai
Copy link
Member

Sooory, my bad :-). I forgot that the working directory was write protected, so 3dfier was not allowed to write the output.

But indeed, 3dfier fails silently in such chase.

@vvmruder
Copy link
Contributor Author

Maybe we could improve this. It is especially disturbing if all process runs through and in the end when it tries to write the output like CityJSON it fails silently and you are left with no output. After all the time of worried waiting :)

@balazsdukai
Copy link
Member

Indeed, feel free to add your thoughts to #95

@vvmruder
Copy link
Contributor Author

So process is running as expected now? With PostGIS?

@balazsdukai
Copy link
Member

Unfortunately no. It seems that GDAL was not compiled with the Postgres driver, or the build didn't work out the way it was supposed to. In case of a successful build one should get the following output:

$ ogrinfo --formats | grep 'PostgreSQL -vector-'
  PostgreSQL -vector- (rw+): PostgreSQL/PostGIS

But I get nothing for:

docker run --rm -it tudelft3d/3dfier:latest ogrinfo --formats | grep 'PostgreSQL -vector-'

@vvmruder
Copy link
Contributor Author

Oh ...

I used this https://github.com/tudelft3d/3dfier/pull/88/files#diff-3254677a7917c6c01f55212f86c57fbfR376 to enable it while building. But it seems not to work. So need some time to investigate. Do you found out something meanwhile?

@balazsdukai
Copy link
Member

No, I haven't looked into this further.

@vvmruder
Copy link
Contributor Author

I've updated it to work as expected. Could you please try it again?

vvmruder
vvmruder commented Nov 13, 2019
View reviewed changes
Dockerfile Outdated Show resolved Hide resolved
balazsdukai
balazsdukai reviewed Nov 13, 2019
View reviewed changes
Copy link
Member

@balazsdukai balazsdukai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far so good! Tested the build on the master, develop and feature/stats branches, because I know that these use LASTools instead of libLAS.

I think the size of the image is all right, esp. considering how many libraries are bundled into it.

Also updated the instruction in the documentation and removed some obsolete build scripts.

But before merging, I'm gonna update the travis build script to build and deploy the docker images just like in case of val3dity. Or if you are up for it, please go ahead @vvmruder :-) .

@vvmruder
Copy link
Contributor Author

I will.

@balazsdukai Did you read my comment on commit?

@balazsdukai balazsdukai changed the title [WIP] add alpine docker image definition Add alpine docker image definition Nov 13, 2019
@balazsdukai balazsdukai merged commit ee7cec0 into tudelft3d:development Nov 13, 2019
@vvmruder vvmruder deleted the integrate_with_docker branch November 13, 2019 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@balazsdukai balazsdukai balazsdukai left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vvmruder @balazsdukai @KalleBlomquist